Univariate Time Series Forecasting Research Articles

A Comprehensive Machine and Deep Learning Approach for Aerosol Optical Depth Forecasting: New Evidence from the Arabian Peninsula

AbstractAccurate forecasting of environmental pollution indicators holds significant importance in diverse fields, including climate modeling, environmental monitoring, and public health. In this study, we investigate a wide range of machine learning and deep learning models to enhance Aerosol Optical Depth (AOD) predictions for the Arabian Peninsula (AP) region, one of the world’s main dust source regions. Additionally, we explore the impact of feature extraction and their different types on the forecasting performance of each of the proposed models. Preprocessing of the data involves inputting missing values, data deseasonalization, and data normalization. Subsequently, hyperparameter optimization is performed on each model using grid search. The empirical results of the basic, hybrid and combined models revealed that the convolutional long short-term memory and Bayesian ridge models significantly outperformed the other basic models. Moreover, for the combined models, specifically the weighted averaging scheme, exhibit remarkable predictive accuracy, outperforming individual models and demonstrating superior performance in longer-term forecasts. Our findings emphasize the efficacy of combining distinct models and highlight the potential of the convolutional long short-term memory and Bayesian ridge models for univariate time series forecasting, particularly in the context of AOD predictions. These accurate daily forecasts bear practical implications for policymakers in various areas such as tourism, transportation, and public health, enabling better planning and resource allocation.

Statistical and deep learning models for reference evapotranspiration time series forecasting: A comparison of accuracy, complexity, and data efficiency

Reference evapotranspiration (ETo) is an essential variable in agricultural water resources management and irrigation scheduling. An accurate and reliable forecast of ETo facilitates effective decision-making in agriculture. Although numerous studies assessed various methodologies for ETo forecasting, an in-depth multi-dimensional analysis evaluating different aspects of these methodologies is missing. This study systematically evaluates the complexity, computational cost, data efficiency, and accuracy of ten models that have been used or could potentially be used for ETo forecasting. These models range from well-known statistical forecasting models like seasonal autoregressive integrated moving average (SARIMA) to state-of-the-art deep learning (DL) algorithms like temporal fusion transformer (TFT). This study categorizes monthly ETo time series from 107 weather stations across California according to their length to better understand the forecasting models' data efficiency. Moreover, two forecasting strategies (i.e., recursive and multi-input multi-output) are employed for machine learning and DL models, and forecasts are assessed for different multi-step horizons. Our findings show that statistical forecasting models like Holt-Winters' exponential smoothing perform almost as well as complex DL models. Unlike statistical models, DL models generally suffer from low data efficiency and perform well only when enough data is available. Importantly, although the computational costs of most DL models are higher than statistical methods, this is not the case for all. Considering computational cost, data efficiency, and forecasting accuracy, our findings point to the superiority of the neural basis expansion analysis for interpretable time series forecasting (N-BEATS) architecture for univariate ETo time series forecasting. Moreover, our results suggest Holt-Winters and Theta methods outperform SARIMA – the most employed statistical model for ETo forecasting in the literature – in accuracy and efficiency.

Training LSTMS with circular-shift epochs for accurate event forecasting in imbalanced time series

The use of LSTM models for forecasting has increased substantially because they have proven to yield excellent performance in numerous applications involving time series consisting of normal observations. However, LSTM models tend to perform poorly in forecasting extreme events in imbalanced time series due to the lack of representative extreme event data during training. This study introduces a breakthrough Circular-Shift Epoch Training (CSET) strategy for LSTMs that makes the accurate forecasting of extreme events no more difficult than predicting events in normal time series with similar accuracies. CSET is (a) specifically designed to preserve the natural ordering of observations in epochs during training without any attempt to balance extreme and normal observations, (b) formulated for both univariate and multivariate time series forecasting, (c) applicable to scaler and vector forecasting for the multivariate case and can handle upward, downward, and mixed extremes without any modification, and (d) universal because it can be applied to train LSTMs to forecast events in normal time series or in imbalanced time series in exactly the same manner. Most importantly, the forecasting of normal time series and imbalanced time series do not have to be treated as separate problems. The results from an extensive set of experiments involving forecasting events in simulated and real imbalanced time series confirm that LSTM models trained using CSET dramatically outperform the standard fixed-epoch trained baseline models in (a) forecasting both extreme and normal events and (b) classifying extreme events. Given the ubiquitous use of LSTMs for forecasting, the accurate forecasting of extreme events through the incorporation of the innovative CSET strategy can have a major impact in decreasing the disastrous consequences of dangerous extreme events.

On the update frequency of univariate forecasting models

In univariate time series forecasting, models are typically updated at every single review period. This practice, which includes specifying the optimal form of the model and estimating its parameters, theoretically allows the models to exploit new information and to respond quickly to possible structural breaks. We argue that such updates may be irrelevant in practice, also unnecessarily increasing computational cost and forecast instability. Using two large data sets of monthly and daily series as well as an indicative family of conventional time series models, we investigate several model updating scenarios, ranging from complete model form specification and parameter estimation at every review period to no updating at all. We find that intermediate updating scenarios, including the re-estimation of specific parameters but not necessarily the specification of the model form, can result in similar or even better accuracy with significantly lower computational cost. We also show that similar conclusions hold true for popular machine learning methods, as well as for setups where different approaches are utilized for training the models or accelerating their specification and estimation. We discuss the implications of our findings for manufacturers, suppliers, and retailers and propose avenues for future advances in the area of model frequency updating.

Elevating Univariate Time Series Forecasting: Innovative SVR-Empowered Nonlinear Autoregressive Neural Networks

Efforts across diverse domains like economics, energy, and agronomy have focused on developing predictive models for time series data. A spectrum of techniques, spanning from elementary linear models to intricate neural networks and machine learning algorithms, has been explored to achieve accurate forecasts. The hybrid ARIMA-SVR model has garnered attention due to its fusion of a foundational linear model with error correction capabilities. However, its use is limited to stationary time series data, posing a significant challenge. To overcome these limitations and drive progress, we propose the innovative NAR–SVR hybrid method. Unlike its predecessor, this approach breaks free from stationarity and linearity constraints, leading to improved model performance solely through historical data exploitation. This advancement significantly reduces the time and computational resources needed for precise predictions, a critical factor in univariate economic time series forecasting. We apply the NAR–SVR hybrid model in three scenarios: Spanish berry daily yield data from 2018 to 2021, daily COVID-19 cases in three countries during 2020, and the daily Bitcoin price time series from 2015 to 2020. Through extensive comparative analyses with other time series prediction models, our results substantiate that our novel approach consistently outperforms its counterparts. By transcending stationarity and linearity limitations, our hybrid methodology establishes a new paradigm for univariate time series forecasting, revolutionizing the field and enhancing predictive capabilities across various domains as highlighted in this study.

Short term water demand forecast modelling using artificial intelligence for smart water management

Water is an important resource for life and its existence. Water demand is increasing with increasing economic growth and population, while the water availability is continually depleting making an increasing stress on freshwater resources, necessitating monitoring of water consumption. In addition to controlling the water supply with an efficient water management system, automating the system in terms of both monitoring and operation has received a lot of attention in recent years. Short-term water demand forecast aids in the optimal control of a water supply system and its accurate forecasting helps in reducing operating costs and saving energy. Despite extensive research, the use of demand forecasting for efficient water management has yet to be implemented in India. As a result, the focus of this research is primarily on the model for forecasting short-term water demand using artificial intelligence. A comparative study has been carried out between nine machine learning and deep learning models using the water consumption data over the period from 2020 to 2021 for the city of Hubli in Karnataka. Univariate and multivariate time series forecasting models were considered using the 10-min interval flow meter readings to find the most suitable predictive model. For univariate time series forecasting, only the water consumption was used to predict the water demand, whereas, for the multivariate model, climatic parameters, and calendar inputs (like an hour of the day, holidays, etc.) were considered along with the water consumption data. The results suggest that the deep learning models outperformed the machine learning models, and Long-Short Term Memory (LSTM) model demonstrates the best prediction performance in the two scenarios with a mean absolute error of 0.11 m3/hr for univariate model and 2.96 m3/hr for the multivariate model. The best predictive model can be used to forecast the short-term water demand for any region to ensure sustainable water resource management.

Time series and regression methods for univariate environmental forecasting: An empirical evaluation

One of the most common and valuable applications of science to the environment is to forecast the future, as it affects human lives in many aspects. However, it is not yet clear which methods –conventional time series or regression– deliver the highest performance in univariate time series forecasting. This study attempts to answer that question with a large-scale comparative evaluation that includes 68 environmental variables over three frequencies (hourly, daily, monthly), forecasted in one to twelve steps into the future, and evaluated over six statistical time series and fourteen regression methods. Results suggest that the strongest representatives of the time series methods (ARIMA, Theta) exhibit high accuracies, but certain regression methods (Huber, Extra Trees, Random Forest, Light Gradient Boosting Machines, Gradient Boosting Machines, Ridge, Bayesian Ridge) deliver even more promising results for all forecasting horizons. Finally, depending on the specific use case, the suitable method should be employed, as certain methods are more appropriate for different frequencies and some have an advantageous trade-off between computational time and performance.

Data augmentation for univariate time series forecasting with neural networks

Neural networks have been proven particularly accurate in univariate time series forecasting settings, requiring however a significant number of training samples to be effectively trained. In machine learning applications where available data are limited, data augmentation techniques have been successfully used to generate synthetic data that resemble and complement the original train set. Since the potential of data augmentation has been largely neglected in univariate time series forecasting, in this study we investigate nine data augmentation techniques, ranging from simple transformations and adjustments to sophisticated generative models and a novel upsampling approach. We empirically evaluate the impact of data augmentation on forecasting accuracy considering both shallow and deep feed-forward neural networks and time series data sets of different sizes from the M4 and the Tourism competitions. Our results suggest that certain data augmentation techniques that build on upsampling and time series combinations can improve forecasting performance, especially when deep networks are used. However, these improvements become less significant as the initial size of the train set increases.

A Model for the Prediction of Lifetime Profit Estimate of Dairy Cattle (Student Abstract)

In livestock management, the decision of animal replacement requires an estimation of the lifetime profit of the animal based on multiple factors and operational conditions. In Dairy farms, this can be associated with the profit corresponding to milk production, health condition and herd management costs, which in turn may be a function of other factors including genetics and weather conditions. Estimating the profit of a cow can be expressed as a spatio-temporal problem where knowing the first batch of production (early-profit) can allow to predict the future batch of productions (late-profit). This problem can be addressed either by a univariate or multivariate time series forecasting. Several approaches have been designed for time series forecasting including Auto-Regressive approaches, Recurrent Neural Network including Long Short Term Memory (LSTM) method and a very deep stack of fully-connected layers. In this paper, we proposed a LSTM based approach coupled with attention and linear layers to better capture the dairy features. We compare the model, with three other architectures including NBEATs, ARIMA, MUMU-RNN using dairy production of 292181 dairy cows. The results highlight the performence of the proposed model of the compared architectures. They also show that a univariate NBEATs could perform better than the multi-variate approach there are compared to. We also highlight that such architecture could allow to predict late-profit with an error less than 3$ per month, opening the way of better resource management in the dairy industry.

Early Estimation of Daily Reference Evapotranspiration Using Machine Learning Techniques for Efficient Management of Irrigation Water

Assessing the right amount of water needs for a specific crop is a key task for farmers and agronomists to achieve efficient and optimal irrigation scheduling, and then an optimal crop yield. To this end, the reference evapotranspiration (ET0) was developed. It represents the atmospheric evaporation demand, and therefore an important variable for irrigation management. In this regard, several methods such as the FAO’s Penman-Monteith and Hargreaves have been used to model and estimate ET0. These methods use climatic parameters data for calculation procedures such as solar net radiation (Rn), saturation vapour pressure(es), and min-max air temperatures or a combination of them. In this paper, we investigated two proposed data-driven methods to predict ET0 values in a semi-arid region in Morocco. The first approach is based on forecasting techniques and the second one uses end-to-end modeling of ET0 based on meteorological data and machine learning models. The feature selection and engineering results show that solar global radiation (Rg) and mean air temperature (Ta) have a significance of more than 87% as relevant predictors features for the ET0. We then used them as input to machine learning regression models. Regression evaluation metrics showed that The XGboost regressor model performs well in both cross-validation with R2=0.93 in the first fold, and in hold-out validation with R2=0.92 and RMSE=0.55. As a final step, we compared the univariate time series forecasting of ET0 using the Facebook Prophet model versus the machine learning modeling method that we proposed. As goodness-of-fit measures, forecasting using machine learning modeling of ET0 showed better results in terms of both R2 and RMSE.

An Efficient Multivariate Autoscaling Framework Using Bi-LSTM for Cloud Computing

With the rapid development of 5G technology, the need for a flexible and scalable real-time system for data processing has become increasingly important. By predicting future resource workloads, cloud service providers can automatically provision and deprovision user resources for the system beforehand, to meet service level agreements. However, workload demands fluctuate continuously over time, which makes their prediction difficult. Hence, several studies have proposed a technique called time series forecasting to accurately predict the resource workload. However, most of these studies focused solely on univariate time series forecasting; in other words, they only analyzed the measurement of a single feature. This study proposes an efficient multivariate autoscaling framework using bidirectional long short-term memory (Bi-LSTM) for cloud computing. The system framework was designed based on the monitor–analyze–plan–execute loop. The results obtained from our experiments on different actual workload datasets indicated that the proposed multivariate Bi-LSTM exhibited a root-mean-squared error (RMSE) prediction error 1.84-times smaller than that of the univariate one. Furthermore, it reduced the RMSE prediction error by 6.7% and 5.4% when compared with the multivariate LSTM and convolutional neural network-long short-term memory (CNN-LSTM) models, respectively. Finally, in terms of resource provisioning, the multivariate Bi-LSTM autoscaler was 47.2% and 14.7% more efficient than the multivariate LSTM and CNN-LSTM autoscalers, respectively.

Arima Model Optimal Selection for Time Series Forecasting

Abstract A fast-and-flexible method of ARIMA model optimal selection is suggested for univariate time series forecasting. The method allows obtaining as-highly-accurate-as-possible forecasts automatically. It is based on effectively finding lags by the autocorrelation function of a detrended time series, where the best-fitting polynomial trend is subtracted from the time series. The forecasting quality criteria are the root-mean-square error (RMSE) and the maximum absolute error (MaxAE) allowing to register information about the average inaccuracy and worst outlier. Thus, the ARIMA model optimal selection is performed by simultaneously minimizing RMSE and Max-AE, whereupon the minimum defines the best model. Otherwise, if the minimum does not exist, a combination of minimal-RMSE and minimal-MaxAE ARIMA models is used.

On projection methods for functional time series forecasting

Two nonparametric methods are presented for forecasting functional time series (FTS). The FTS we observe is a curve at a discrete-time point. We address both one-step-ahead forecasting and dynamic updating. Dynamic updating is a forward prediction of the unobserved segment of the most recent curve. Among the two proposed methods, the first one is a straightforward adaptation to FTS of the k-nearest neighbors methods for univariate time series forecasting. The second one is based on a selection of curves, termed the curve envelope, that aims to be representative in shape and magnitude of the most recent functional observation, either a whole curve or the observed part of a partially observed curve. In a similar fashion to k-nearest neighbors and other projection methods successfully used for time series forecasting, we “project” the k-nearest neighbors and the curves in the envelope for forecasting. In doing so, we keep track of the next period evolution of the curves. The methods are applied to simulated data, daily electricity demand, and NOx emissions and provide competitive results with and often superior to several benchmark predictions. The approach offers a model-free alternative to statistical methods based on FTS modeling to study the cyclic or seasonal behavior of many FTS.

A deep learning approach to solar radio flux forecasting

The effect of atmospheric drag on spacecraft dynamics is considered one of the predominant sources of uncertainty in Low Earth Orbit. These effects are characterised in part by the atmospheric density, a quantity highly correlated to space weather. Current atmosphere models typically account for this through proxy indices such as the F10.7, but with variations in solar radio flux forecasts leading to significant orbit differences over just a few days, prediction of these quantities is a limiting factor in the accurate estimation of future drag conditions, and consequently orbital prediction. In this work, a novel deep residual architecture for univariate time series forecasting, N-BEATS, is employed for the prediction of the F10.7 solar proxy on the days-ahead timescales relevant to space operations. This untailored, pure deep learning approach has recently achieved state-of-the-art performance in time series forecasting competitions, outperforming well-established statistical, as well as statistical hybrid models, across a range of domains. The approach was found to be effective in single point forecasting up to 27-days ahead, and was additionally extended to produce forecast uncertainty estimates using deep ensembles. These forecasts were then compared to a persistence baseline and two operationally available forecasts: one statistical (provided by BGS, ESA), and one multi-flux neural network (by CLS, CNES). It was found that the N-BEATS model systematically outperformed the baseline and statistical approaches, and achieved an improved or similar performance to the multi-flux neural network approach despite only learning from a single variable.

The Wisdom of the Data: Getting the Most Out of Univariate Time Series Forecasting

Forecasting is a challenging task that typically requires making assumptions about the observed data but also the future conditions. Inevitably, any forecasting process will result in some degree of inaccuracy. The forecasting performance will further deteriorate as the uncertainty increases. In this article, we focus on univariate time series forecasting and we review five approaches that one can use to enhance the performance of standard extrapolation methods. Much has been written about the “wisdom of the crowds” and how collective opinions will outperform individual ones. We present the concept of the “wisdom of the data” and how data manipulation can result in information extraction which, in turn, translates to improved forecast accuracy by aggregating (combining) forecasts computed on different perspectives of the same data. We describe and discuss approaches that are based on the manipulation of local curvatures (theta method), temporal aggregation, bootstrapping, sub-seasonal and incomplete time series. We compare these approaches with regards to how they extract information from the data, their computational cost, and their performance.

Comparative study of data-driven short-term wind power forecasting approaches for the Norwegian Arctic region

This paper conducts a systemic comparative study on univariate and multivariate wind power forecasting for five wind farms inside the Arctic area. The development of wind power in the Arctic can help reduce greenhouse gas emissions in this environmentally fragile region. In practice, wind power forecasting is essential to maintain the grid balance and optimize electricity generation. This study first applies various learning methods for wind power forecasting. It comprehensively compares the performance of models categorized by whether considering weather factors in the Arctic. Nine different representative types of machine-learning algorithms make several univariate time series forecasting, and their performance is evaluated. It is demonstrated that machine-learning approaches have an insignificant advantage over the persistence method in the univariate situation. With numerical weather prediction wind data and wind power data as inputs, the multivariate forecasting models are established and made one to six h in advance predictions. The multivariate models, especially with the advanced learning algorithms, show their edge over the univariate model based on the same algorithm. Although weather data are mesoscale, they can contribute to improving the wind power forecasting accuracy. Moreover, these results are generally valid for the five wind farms, proving the models' effectiveness and universality in this regional wind power utilization. Additionally, there is no clear evidence that predictive model performance is related to wind farms' topographic complexity.

Energy time series forecasting-analytical and empirical assessment of conventional and machine learning models

Machine learning methods have been adopted in the literature as contenders to conventional methods to solve the energy time series forecasting (TSF) problems. Recently, deep learning methods have been emerged in the artificial intelligence field attaining astonishing performance in a wide range of applications. Yet, the evidence about their performance in to solve the energy TSF problems, in terms of accuracy and computational requirements, is scanty. Most of the review articles that handle the energy TSF problem are systematic reviews, however, a qualitative and quantitative study for the energy TSF problem is not yet available in the literature. The purpose of this paper is twofold, first it provides a comprehensive analytical assessment for conventional, machine learning, and deep learning methods that can be utilized to solve various energy TSF problems. Second, the paper carries out an empirical assessment for many selected methods through three real-world datasets. These datasets related to electrical energy consumption problem, natural gas problem, and electric power consumption of an individual household problem. The first two problems are univariate TSF and the third problem is a multivariate TSF. Compared to both conventional and machine learning contenders, the deep learning methods attain a significant improvement in terms of accuracy and forecasting horizons examined. In the meantime, their computational requirements are notably greater than other contenders. Eventually, the paper identifies a number of challenges, potential research directions, and recommendations to the research community may serve as a basis for further research in the energy forecasting domain.

Impact of COVID-19 Pandemic on Thailand Economy and Tourist Industry: An Empirical Forecasting Using ARIMA

Goal: The tourism sector always plays a vital role among other sectors, which contributes heavily to Thailand’s GDP. Due to the recent Pandemic crises, the tourism industry worldwide faces a decline.  The aim of writing this paper is to forecast, tourist inflow for Thailand for the next nine months by using ARIMA forecasting
 Design/Methodology/Approach: In order to meet the study objective, we adopted the univariate time series forecasting of tourist arrival. The AR(12) and MA(12) process with d(1) order to specify the appropriate model also applied for forecasting selection of international tourist arrivals, after diagnostic checking of series through correlogram Q-statistics and correlogram of residuals for adjusted ARIMA (12,1,12) in which we found low residual against 70 lags.
 Results: The evidence predicts that in the next nine months, Thailand will face significant negative zone arrival of international tourists due to the COVID-19 pandemic crises, which adversely affect Thailand’s economy due to the shortfall of international tourist arrivals. However decline in tourist arrival will eventually decrease air pollution in Thailand, thus it positively impact on environmental quality.
 Limitation of the investigation: In terms of limitation of this study, this forecasting is only valid for Thailand based on historical and current data.
 Practical implications: We highly recommend government of Thailand for enchasing overall logistics performance and furthermore increase use to renewable energy and green performance in their tourism sector.
 Originality / Value: This study provides tourism forecasting estimation for Thailand, which can help government to reshape their tourism policies due to Covid-19 pandemic crises. As this study is the first attempt to provide forecasting related with Covid-19 for tourism sector.

A machine learning approach to univariate time series forecasting of quarterly earnings

We propose our quarterly earnings prediction (QEPSVR) model, which is based on epsilon support vector regression (ε-SVR), as a new univariate model for quarterly earnings forecasting. This follows the recommendations of Lorek (Adv Account 30:315–321, 2014. https://doi.org/10.1016/j.adiac.2014.09.008), who notes that although the model developed by Brown and Rozeff (J Account Res 17:179–189, 1979) (BR ARIMA) is advocated as still being the premier univariate model, it may no longer be suitable for describing recent quarterly earnings series. We conduct empirical studies on recent data to compare the predictive accuracy of the QEPSVR model to that of the BR ARIMA model under a multitude of conditions. Our results show that the predictive accuracy of the QEPSVR model significantly exceeds that of the BR ARIMA model under 24 out of the 28 tested experiment conditions. Furthermore, significance is achieved under all conditions considering short forecast horizons or limited availability of historic data. We therefore advocate the use of the QEPSVR model for firms performing short-term operational planning, for recently founded companies and for firms that have restructured their business model.

Forecasting Economic Time Series in the Presence of Variance Instability and Outliers

This work examines the impact of data transformation (for variance stabilization) and outlier adjustment (“linearization”) on the quality of univariate time series forecasts, considering each one separately, as well as in combination. Twenty of the most important time series of the Greek economy were used for this purpose. Empirical findings show a significant improvement in forecasts’ confidence intervals, but no substantial improvement in point forecasts. Furthermore, the combined transformation-linearization procedure improves substantially the non-normality problem encountered in many macroeconomic time series.